🎯 Goal: To introduce the concept of AI as a collaboration tool, and AI agents as a powerful new tool for improving research workflows and to demonstrate their practical application within a secure virtual lab environment, using the study of hospital mergers as a case study.
📝 Note: I created a deep research article on this topic: https://docs.google.com/document/d/1_pBhzXYG1cHE1g6oS3SL1F_Nete-KrddQYG0eTP0i4Q/edit?usp=sharing
📚 Traditional Workflow: Manual literature reviews, Researcher driven coding
🚀 New Wave: LLM-powered AI Augmentation: AI as collaborative partner, mostly through a chat interface. Similar to early days of “googling”
🎯 Next wave: LLM powered by tools, e.g. AI agents: AI Agents as workflow enablers
🔍 What are these new tools?
AI Agents are mostly autonomous systems built on LLMs by giving them access to tools that can:
🎯 Manage context, e.g. using memory to keep track of previous steps
📝 Create detailed plans
🛠️ Use various tools (e.g. web search, code execution, data analysis, write reports)
🔄 Execute multi-step tasks
💬 Chatbots : Basic Q&A capabilities, single-turn responses, limited context retention
🧠 Reasoning Engines : Multi-step thinking, improved context handling, still confined in scope
🚀 AI Agents : Proactive problem-solving, advanced tool utilization, autonomous goal pursuit
💡 Key Analogy: Think of an AI Agent as a research assistant you can delegate tasks to, not just a calculator for computations.
📚 Automated Literature Synthesis : Comprehensive source scanning, Automated citation management, Intelligent synthesis
💻 Code Generation & Debugging : Natural language to code, Multi-language support (Python, R, Stata), Intelligent debugging
🔍 Autonomous Data Handling : Automated data collection, Smart cleaning algorithms, Advanced analysis tools, Multi-source integration
🔄 Workflow Orchestration : End-to-end automation, Simulation management, Results analysis, Visualization generation
💻 Development Environment: Modern IDEs integration, Seamless workflow integration, Real-time assistance
🤖 Agent Interaction : Chat-based interface, Customizable personas, Task-specific fine-tuning, Economic research specialization
🌐 Available Platforms : 📓 Notebook LM with research tools, 🔧 Gemini/OpenAI/Anthropic integration, 🛠️ VSCode with extensions (e.g., Clive), 🚀 Cursor’s built-in agent support, AWS bedrock, Azure openai, Google vertex ai, Python frameworks like langchain.
📊 Deep research : Gemini, Claude, OpenAI all have their own ways to do deep research. Its basically where you have an LLM attached to tool, you instruct it what to do then it returns with a report. I have used Gemini for deep research, the most. Use it for the planning phase of a tool build.
Tools are anything which the LLM has access to which it can use to complete a task
MCP is how LLMs access different tools, think of it as an api wich allows the LLM to interact with something
Context refers to the memory of an LLM. There are often limits to the context of different models
Hallucinations are more commond as we fill up the tokens the model is tracking.
LLM and agents excell at well scoped task where full context is given, creating a plan before executing is important to think through the task and the results.
Documenataion and githhub can add context and add value to the agents
🤔 Trust but verify. When creating a plan, always think about how to verify the results.
Do we have a dataset we trust? Then we have labeled data we can test against and create score against.
Create a test data to run a simulation to unit test the model being developed.
As we thinking about what we want agents to do, we need to think about how to evaluate them.
Having general benchmarkets is important to help us understand the performance of the agents across different models and domains.
LLMs from big providers collect your data and we need to be careful about what is shared.
⚠️ Security Challenges : Public AI tool risks, data exposure concerns, privacy compliance, regulatory requirements
💪 AWS Bedrock Solution : Private cloud environment, leading LLM access, secure infrastructure, controlled data flow
✅ Key Guarantees : No external data training, complete confidentiality, virtual private cloud, data integrity preservation
🎯 End Result : Powerful AI utilization, zero compromise on security, full research capability, regulatory compliance
A research team evaluates economic impacts of a proposed hospital merger using AI-powered analysis.
🔄 AI-Assisted Workflow
📝 Initial Task Delegation : High-level goal setting and planning. Human expert provides high level of oversight. Can use tools to help find ambiguities in the task. High level of payoff for human involvement. How will we test?
🔍 Data Collection Phase : 📊 Regulatory and news filing data web scraping via a script. Get a test dataset to compare results against.
🤖 Model Development and teaching : Read this paper, code and help me implment model. What are the tradeoffs? Help me create a simulation using test data to understand the model.
📈 Analysis & Visualization : 📊 Write analysis code from task delegation, weigh tradeoffs from using different open source tools, 📉 Trend visualization, 📑 Comprehensive reporting
Source: Nature Paper | GitHub Repository
🚀 Transformation: Researchers are becoming managers adding value by providing expertise and guidance to the agents.
🔒 Security & Trust : Public providers keep your data. AWS Bedrock integration allows us to protect our data and IP
💪 Empowerment: Knowing the what you want to do and how to verify lowers the barrier to using new toolset
AI Agents & Secure Virtual Labs